Thereason thatthemethod MSO in"Efficient multi-objectivemolecular optimization inacontinuous3 latent space" achieved ahigher penalized logP with unlimited property evaluations than ours (26.1 vs 15.18) isdue4 to different experimental settings. With a8 largerLmax, the best penalized logP score can be significantly increased. Wehavestarted11 running the experiments on GuacaMol as suggested. We will fix these two figures in the final version. All generated molecules in the appendix have been24 double-checked by both RDkit and human experts.

How much data is required to learn the structure of a language via next-token prediction?

Understanding how the structure of language can be learned from sentences alone is a central question in both cognitive science and machine learning. Studies of the internal representations of Large Language Models (LLMs) support their ability to parse text when predicting the next word, while representing semantic notions independently of surface form. Yet, which data statistics make these feats possible, and how much data is required, remain largely unknown. Probabilistic context-free grammars (PCFGs) provide a tractable testbed for studying these questions. However, prior work has focused either on the post-hoc characterization of the parsing-like algorithms used by trained networks; or on the learnability of PCFGs with fixed syntax, where parsing is unnecessary. Here, we (i) introduce a tunable class of PCFGs in which both the degree of ambiguity and the correlation structure across scales can be controlled; (ii) provide a learning mechanism -- an inference algorithm inspired by the structure of deep convolutional networks -- that links learnability and sample complexity to specific language statistics; and (iii) validate our predictions empirically across deep convolutional and transformer-based architectures. Overall, we propose a unifying framework where correlations at different scales lift local ambiguities, enabling the emergence of hierarchical representations of the data.

Table 10 presents the hyperparameters used to train dPads programmatic models.

T op left: The agent's perspective at initialization. The agent does not see the orange sphere, and so will have to search for it.